A Reformulation of the Semantic Gap Problem in Content-Based Image Retrieval Scenarios

Abstract This paper considers the notion of the “semantic gap” problem – i.e. how to enable a machine to recognize the semantic properties of an image – as it is commonly formulated in the domain of content-based image retrieval. Drawing on ethnographic studies of design professionals who routinely engage in image search tasks we seek to demonstrate the means by which aesthetic and affective concepts become associated with images and elements of images within a cooperative design process of selection, discussion and refinement and how these often do not correspond to the unused semantic tags provided in image libraries. We discuss how we believe the problem of the semantic gap is misconstrued and discuss some of the technology implications of this.